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Relationships  Among  Standardized  Literacy  Test  Scores, 
ROTC  Grades,  and  Officer  Basic  Course  Performance 

Introduction 


There  have  been  increasing  concerns  that  junior  officers  lack 
adequate  literacy  skills  to  perform  their  jobs  effectively.  In  response 
to  a  1986  request  from  the  Commanding  General,  Training  and  Doctrine 
Command  (TRADOC),  the  Deputy  Chief  of  Staff  for  Operations  and  Plans 
formed  the  Precommissioning  Literacy  Skills  Standards  Task  Force  to 
assess  the  feasibility  of  establishing  literacy  skills  standards  for 
implementation  in  all  sources  of  commissioning. 

The  Task  Force  has  accomplished  its  initial  mission  with  the 
formulation  of  the  Precommissioning  Literacy  Standards  Pilot  Program, 
designed  by  the  Army  Writing  Program,  TRADOC.  The  basic  approach  of 
this  program  is  to  provide  (l)  diagnostic  screening  of  communication 
skills  in  speaking,  writing,  and  reading,  (2)  developmental  teaching  to 
cadets  with  deficiencies  in  these  skills,  and  (3)  subsequent  assessment 
of  their  developmental  progress.  The  literacy  skills  assessment  will  be 
aimed  at  diagnosis  and  re-evaluation,  not  at  eliminating  cadets  with 
inadequate  skills  from  commissioning. 

Much  of  the  assessment  and  developmental  teaching  procedures  will 
be  incorporated  into  the  precommissioning  training  curriculum  in  order 
to  provide  job-appropriate  contents  and  to  maximize  the  utility  of  the 
limited  training  hours.  The  pilot  program  will  include  one  standardized 
test  of  writing  to  be  used  as  one  of  the  diagnostic  screening 
instruments.  Many  aspects  of  the  program  will  remain  tentative,  pending 
evaluation  of  the  pilot  program  outcomes  and  a  thorough  examination  of 
the  social,  political,  and  organizational  ramifications  of  such  a 
program. 

Crucial  steps  in  developing  literacy  skills  standards  are  the 
selection  of  methods  for  assessing  literacy  skills  and  the  assessment  of 
the  accuracy  and  reliability  of  the  selected  methods.  Standardized 
tests  of  reading/writing  provide  important  benefits.  Generally,  the 
tests  are  quite  reliable.  They  minimize  errors  in  evaluating 
individuals'  skill  levels  that  are  due  to  non-standardized  test  contents 
or  variations  in  administration  and  scoring  procedures  across  divergent 
institutions  in  different  parts  of  the  country. 

However,  the  objective  of  establishing  the  precommissioning 
literacy  skills  standards  would  be  to  ensure  that  all  newly  commissioned 
officers  possess  sufficient  literacy  skills  to  perform  officer  duties 
satisfactorily.  Thus,  in  addition  to  reliability,  the  relevance  of 
skills  measured  by  standardized  tests  to  officer  performance  needs  to  be 
addressed.  This  examination  is  especially  critical  in  view  of  the 
possible  drawbacks  that  may  accompany  use  of  standardized  tests  such  as 
purchase  and  scoring  costs,  personnel  resources  required,  impact  on 
program  hours  available  for  training,  and  social/ political/ policy 
implications  of  performance  differences  by  racial/ ethnic  groups  which 
tend  to  emerge  in  large-scale  testing  of  basic  skills. 


1 


As  one  of  the  investigative  steps,  the  Task  Force  requested  Army 
Research  Institute  (ARl)  to  analyze  data  available  at  ARI  and  relevant 
to  the  Task  Force  mission.  The  purpose  of  this  report  is  to  describe 
the  results  of  the  analyses  which  focused  on  the  relationships  between 
skills  measured  by  standardized  literacy  tests  and  performance  in 
various  aspects  of  officer  training.  The  analysis  approach  was  based  on 
the  assumption  that  standardized  tests  would  be  used  as  diagnostic 
instruments . 

The  specific  questions  addressed  were: 

1 .  Do  cadets  perform  at  similar  levels  on  different  standardized  tests? 
If  so,  the  findings  reported  below  might  generally  apply  to  other 
standardized  tests  such  as  the  one  to  be  included  in  the  pilot  program. 
In  addition,  test  scores  commonly  available  in  cadet  records  (e.g.,  SAT) 
may  be  substituted  for  scores  on  a  different  test  used  for  literacy 
assessment. 

2.  How  much  do  literacy  test  scores  overlap  with  precommissioning 
officer  training  performance?  If  cadets  have  literacy  problems,  do  they 
tend  to  have  problems  in  other  aspects  of  officer  training? 

5.  How  well  do  literacy  test  scores  predict  performance  in  Officer 
Basic  Courses  (OBCs)?  Do  cadets  with  relatively  lower  test  scores 
perform  in  the  lower  range  in  OBC? 

4.  To  what  extent  can  OBC  performance  be  predicted  from 
precommissioning  performance  measures  including  standardized  literacy 
skills  tests? 


Procedure 


Data  Used  for  Analysis 

Data  generated  from  the  following  three  sources  were  analyzed: 

(1)  Army  Reserve  Officers’  Training  Corps  (ROTC)  Achievement  Testing 
Program,  (2)  ROTC  Advanced  Camp  and  Commissioning  Files,  and  (3)  OBC 
grades  from  Armor,  Engineer,  and  Infantry  Schools. 

ROTC  Achievement  Testing  Program  data.  Standardized  tests  of 
writing  and  reading  were  administered  to  Military  Science  (MS)  I  and  MS 
IV  cadets  ROTC-wide  during  the  school  years  1983-84,  1984-85,  and 
1985-86.  For  the  purpose  of  this  report,  data  on  Missouri  College 
English  Test  (MCET)  (Callis  &  Johnson,  1965)  and  Nelson-Denny  Reading 
Test  (NELSON-DENNY)  (Brown,  Bennett,  &  Hanna,  1981)  from  the  first  two 
school  years  were  analyzed. 

To  assess  skill  levels  by  using  tests,  raw  test  scores  alone  does 
not  provide  enough  information.  For  example,  the  meaning  of  a  raw  score 
of  112  on  NELSON-DENNY,  which  has  a  raw  score  range  of  0-172,  is  not 
immediately  clear.  One  method  for  interpreting  raw  scores  is  to  compare 
an  individual's  score  against  scores  made  by  a  large  group  of 
individuals.  This  method  translates  raw  scores  into  indices  of 
performance  relative  to  the  comparison  group. 

For  an  earlier,  detailed  analysis  of  these  data  (Hunter,  1986), 
cadets'  test  scores  were  converted  to  ROTC  percentile  scores.  These 
percentile  scores  provide  two  advantages  for  the  present  analyses: 
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(1 )  current,  age-appropriate  comparison  group  and  (2)  cadets  scores 
expressed  in  terras  of  the  approximate  percentage  of  ROTC  cadets  whose 
performance  they  surpassed.  Referring  hack  to  the  raw  score  of  112  on 
the  NELSON-DENNY ,  the  corresponding  ROTC  percentile  score  for  MS  IV 
cadets  is  38.  Cadets  with  the  score  of  112  surpassed  38^  of  MS  IV 
cadets  on  this  reading  test. 

The  ROTC  percentile  scores  were  based  on  the  raw  scores  generated 
by  7,300-7,500  MS  IV  cadets  (depending  on  the  test)  participating  in  the 
ROTC  Achievement  Testing  Program  each  year.  This  constituted  about  90% 
or  greater  segment  of  the  entire  MS  IV  population.  The  “Composition  of 
these  data  sets  in  terms  of  gender  and  ROTC  region  representations 
approximated  those  of  the  entire  MS  IV  population  as  reported  in  the 
ROTC  Enrollment  Reports  of  the  respective  years  (US  Army  Training  and 
Doctrine  Command,  1983,  1984)* 

ROTC  Advanced  Camp  and  Commissioning  File  data.  The  ROTC  Advanced 
Camp  is  a  6-week  training  exercise,  designed  by  the  US  Array  Infantry 
School  to  provide  cadets  with  opportunities  to  apply  their 
technical/tactical  knowledge  and  leadership  skills  in  simulated  tactical 
situations.  Generally,  the  Advanced  Camp  is  attended  by  cadets  between 
their  jxinior  and  senior  years  in  college,  i.e.,  between  Military  Science 
(ms)  III  and  MS  IV  courses.  They  are  evaluated  on  various  military 
skills  and  physical  readiness,  but  the  main  purpose  of  the  camp  is  the 
application  and  development  of  tactical  and  leadership  skills. 

Data  from  the  Advanced  Camp  in  1982,  1983,  and  1984  were  used 
consisting  of  approximately  9>000  cases  per  year.  From  the  camp 
records,  summary  scores  for  land  navigation  (LNAV),  rifle  marksmanship 
(rifle),  physical  readiness  test  (PT),  tactical  application  exercises 
(tax),  and  peer  rating  of  cadets'  officer  potential  (PEER)  were 
examined.  These  data  were  selected  as  possible  indicators  of  various 
key  components  of  officer  potential. 

The  TAX  scores  are  ratings  by  the  squad  officer  advisor/ trainer 
(scat)  based  on  cadets’  performance,  both  as  squad  leaders  and  members, 
during  the  simulated  tactical  application  exercises.  The  summary  TAX 
score  is  a  sum  of  the  following  nine  ratings:  supervision  and  control 
of  subordinates,  ability  to  work  as  a  squad  member,  verbal  and 
non-verbal  communication  skills,  decisiveness,  technical  and  tactical 
proficiency,  mission-oriented  attitude /motivation,  confidence  in  own 
actions,  effective  planning,  and  mission  accomplishment.  Individuals' 
ratings  across  the  nine  TAX  components  are  closely  associated  (internal 
consistency:  alpha  —  .97)*  Each  cadet  selects  the  top  10  and  the  lowest 
10  cadets  from  his/her  platoon  with  respect  to  officer  potential.  Based 
on  these  rankings,  one  composite  peer  rating  for  each  cadet  is 
calculated. 

The  ROTC  Commissioning  Files  contain  personnel  data  of  cadets  who 
completed  the  precommissioning  training  and  were  commissioned.  From  the 
Commissioning  Files  of  1983,  1984,  and  1985,  containing  approximately 
8,200  cases  per  year,  the  following  data  were  examined:  cumulative 
undergraduate  academic  grade  point  average  (GPA) ,  cumulative  Military 
Science  grade  point  average  (ROTC),  and  Scholastic  Aptitude  Test  (SAT) 
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and  American  College  Testing  Program  Examination  (ACT)  scores  when 
available.  (Both  college  and  Military  Science  GPAs  are  coded  in  uniform 
scales  across  all  ROTC  detachments.) 

It  should  be  noted  that  the  precommissioning  measures  listed  above 
would  reflect  varying  degrees  of  literacy  skills.  For  example,  the 
academic  average  would  consist  largely  of  tests  taken  in  college  courses 
most  of  which  would  involve  at  least  some  literacy  skills.  On  the  other 
hand,  some  of  the  Advanced  Camp  grades  may  mostly  represent  abilities 
other  than  literacy  skills  which  are,  nonetheless,  essential  for 
effective  officer  performance. 

OBC  grades .  Grades  from  the  Armor,  Engineer,  and  Infantry  Officer 
Basic  Courses,  extracted  from  TRADOC's  Automated  Instructional 
Management  System,  were  available.  The  Armor  School  data  covered  15 
classes  (total  of  776  cases)  during  the  period  October  1984  to  November 
1985.  The  Engineer  OBC  data  covered  4  classes  (total  of  513  cases)  from 
June  to  October  1985.  The  Infantry  OBC  data  covered  5  classes  (total  of 
492  cases)  from  July  to  December  1985*  The  three  OBC  data  sets  include 
the  classes  immediately  after  college  graduation  time  as  well  as  classes 
starting  later  in  the  fall.  Each  school  provided  the  final  grade 
(final)  for  each  student,  composed  of  his/her  overall  OBC  performance  in 
technical/military  skills  and  leadership/management  training.  In 
addition,  some  branch-specific  task  subscores  were  available  from  the 
Engineer  and  the  Infantry  Schools. 

The  OBC  final  grades  summarize  how  well  students  performed  in  OBCs 
and  is  interpreted  further  as  their  overall  preparedness  for  officer 
jobs.  On  the  other  hand,  since  most  of  the  scores  included  in  the  final 
grade  are  based  on  paper-and-pencil  tests,  these  grades  may  also  be 
related  to  the  students'  abilities  to  perform  officer  tasks  requiring 
literacy  skills.  It  should  be  noted  that  the  relationship  between  OBC 
grades  and  officer  performance  in  the  field  remains  to  be  examined  and 
is  beyond  the  scope  of  this  report.  Since  the  instructional  contents  of 
the  three  schools  differ,  analyses  involving  OBC  grades  were  performed 
separately  for  each  school. 

Analysis  Approach 

Table  1  summarizes  all  measures  examined  for  this  report  and  the 
total  number  of  cases  available  for  each  measure.  However,  the  number 
of  cases  available  for  specific  analysis  varied  depending  on  the 
measures  being  examined.  About  4,000  records  contained  both  the 
achievement  test  scores  and  the  Advanced  Camp  data.  About  3,000  records 
contained  both  the  test  scores  and  academic  and  ROTC  grade  point 
averages.  For  comparisons  of  the  precommissioning  and  OBC  data,  about 
300  Armor,  90  Engineer,  and  300  Infantry  records  were  available. 
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Table  1 


Label,  Number  of  Cases,  Mean,  Standard 
for  Each  Measure  Analyzed 

Deviation 

,  and  the  Value  Range 

Source  of  Data/Measure 

Label 

No .  of 
Cases 

Mean 

SB 

Range 

ROTC  Achievement  Testing 

Missouri  College  English  Test 
ROTC  Percentile  Score 
of  the  Total  Score 

MCET 

14,802 

49.5 

28.9 

0-99 

Nelson-Denny  Reading  Test 
ROTC  Percentile  Score 
of  the  Total  Score 

NELSON - 
BENNY 

14,512 

49.5 

28.9 

0-99 

Advanced  Camp 

Land  Navigation 

LNAV 

25,710 

86.6 

12.0 

1-100 

Rifle  Marksmanship 

RIFLE 

25,710 

29.5 

5.1 

0-40 

Physical  Readiness  Test 

PT 

25,710 

262.2 

34.1 

o 

o 

1 

Tactical  Application 

Exercise  Total  Score 

TAX 

25,473 

24.9 

7.7 

9-45 

Peer  Rating  of  Officer 
Potential 

PEER 

25,710 

99.9 

17.7 

1-151 

Commissioning  File 

Amer  Coll  Test  Prog  Exam 

ACT 

2,508 

79.1 

29.5 

5-134 

Scholastic  Aptitude  Test 

SAT 

5,563 

1027.9 

180.4 

460-1540 

Academic  Grade  Point  Average 

GPA 

16,482 

27.4 

4.8 

7-40 

ROTC  Military  Science  GPA 

ROTC 

17,959 

34.1 

5.0 

10-40 

Armor  Officer  Basic  Course 
Final  Grade 

FINAL 

776 

93.7 

3.2 

77-99 

Engineer  Officer  Basic  Course 

313 

85.2 

23-98 

Final  Grade 

FINAL 

9.2 

Field  Tactical  Techniques 

FIELD 

269 

84.1 

11.1 

0-99 

Training/ Management 

TRMNG 

238 

91 .4 

6.6 

0-98 

Leadership  -  Advanced 

LDSHP 

238 

84.2 

12.2 

0-100 

(Table  1  continued.) 
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Table  1  continued. 


No.  of 


Source  of  Data/Measure 

Label 

Cases 

Mean 

SD 

Range 

Infantry  Officer  Basic  Course 
Final  Grade 

FINAL 

492 

84.2 

8.2 

18-98 

Training/ Management 

TRMNG 

492 

79.6 

13-6 

20-100 

Vehicle  Maintenance 

MAINT 

492 

87.0 

15.0 

0-99 

Combined  Tactics 

TACTIC 

492 

82.6 

13.9 

0-100 

To  examine  the  degree  of  association  among  measures,  correlational 
analyses  were  performed.  All  measures  used  in  the  correlational 
analyses  were  converted  into  _z  scores.  For  MCET  and  NELSON— DENNY ,  _z 
scores  were  derived  with  reference  to  the  entire  MS  IV  data  sample. 

In  predicting  performance  from  measures  collected  earlier,  the 
number  of  cases  available  for  analyses  tend  to  decrease  over  the  span  of 
time.  This  may  be  due  to  (1 )  natural  selection  and  attrition  of 
applicable  cases  over  time  and/or  (2)  sampling  portions  of  applicable 
cases  for  data  analysis.  Systematic  selection  of  cases  or  reduction  in 
analysis  sample  size  tends  to  restrict  the  variability  of  scores. 
Restriction  in  score  variability,  in  turn,  tends  to  lessen  the  index  of 
predictive  relationship  obtained  from  statistical  analyses. 

In  the  case  of  precommissioning  literacy  assessment,  if  a 
diagnostic  test  is  given  in  the  ROTC  basic  course  before  the  contracting 
selection  occurs,  the  variability  in  test  scores  may  be  greater  than 
that  for  MS  IV  cadets.  The  greater  range  of  predictor  scores  might 
produce  correlations  with  later  measures  (e.g.,  OBC  grades)  that  are 
higher  than  those  calculated  from  MS  IV  test  scores.  However,  for  MCET 
and  NELSON-DENNY,  the  test  score  variability  was  comparable  between  the 
MS  I  and  MS  IV  data  and  also  between  the  MS  IV  population  and  the  OBC 
samples.  Thus,  the  correlations  reported  below  are  judged  to  be 
relatively  unaffected  by  the  restriction  of  test  score  variability 
problem.  However,  such  consistency  in  test  score  variability  may  not  be 
found  for  other  literacy  skills  tests. 

In  addition,  the  training  performance  of  cadets  scoring  in  the 
"low"  score  range  on  the  achievement  tests  was  examined  in  more  detail. 
For  that  examination,  cadet  records  were  grouped  into  the  Low  Score  and 
Higher  Score  groups  according  to  the  ROTC  percentile  scores  on  the  MCET. 
The  25th  percentile  score  was  arbitrarily  chosen  to  divide  the  cadets 
into  these  groups,  since  we  currently  lack  sufficient  empirical 
information  as  to  exactly  what  score  on  this  test  would  represent 
adequate  literacy  skills  for  officer  job  performance.  Dividing  test 
score  distributions  into  quartiles  for  analytical  purposes  is  an 
often-used  convention  and  should  not  be  interpreted  as  having  specific 
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implications  to  decisions  concerning  diagnostic  and/or  selection 
standards. 


Findings 


The  findings  are  described  by  each  of  the  four  major  questions. 
Question  1 

How  consistent  are  cadets'  relative  performance  levels  measured  by 
different  standardized  tests  of  similar  subject  matters?  Though  MCET 
and  NELSON-DENNY  are  not  being  considered  for  use  in  the  proposed 
literacy  skills  assessment,  if  performance  tends  to  be  consistent  across 
these  tests,  then  the  findings  from  the  present  analyses  may  be 
generalized  to  other  standardized  literacy  tests.  In  addition,  strong 
associations  between  some  test  scores  may  suggest  possibilities  for 
using  data  already  available  in  some  cadet  records  (e.g.,  SAT)  as 
indicators  of  literacy  skills. 

Finding  la;  Based  on  the  entire  data  sample,  MCET  and  NELSON-PENNY 
were  highly  correlated.  However,  some  cadets  performed  better  on  MCET 
than  on  NEL SON -DENNY ,  and  vice  versa.  Cadets  who  did  well  on  MCET  were 
likely  to  do  well  on  NELSON-DENNY  (r  =  .73,  Table  2).  This  finding 
suggests  that  the  findings  reported  later  based  on  MCET  would  be 
generally  applicable  to  NELSON-DENNY  and  may  also  apply  to  other 
standardized  tests  of  literacy  skills. 


Table  2 

Correlations  Among  Standardized  Tests  for  the  Total  MS  IV  Sample 


MCET 


NELSON-DENl^Y  SAT 


ACT 


NELSON-DENNY 

.73  (11918) 

SAT 

.64  (  2611) 

.71 

(  2882) 

ACT 

.40  (  1265) 

.45 

(  1288) 

.42  (  270) 

GPA 

.33  (  7522) 

.31 

(  7727) 

.31  (  3946)  .22  (  1876) 

Note:  Numbe 
parentheses  • 


However,  a  correlation  of  .73  also  suggests  that  predicting 
specific  individuals’  performance  from  one  test  to  the  other  would  lead 
to  some  degree  of  error.  Table  3  shows  the  distribution  of  cadets 
grouped  into  low  and  higher  MCET  score  ranges  (25  or  Below  and  26  or 
Above)  by  score  ranges  on  NELSON-DENNY.  The  percentages  are  with 
reference  to  each  of  the  MCET  Score  groups.  66^  of  cadets  who  scored  25 
or  below  on  MCET  also  scored  in  the  same  range  on  NELSON-DENNY. 

However,  34^  of  those  with  MCET  scores  of  25  or  below  produced  higher 
scores  on  NELSON-DENNY. 
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Table  5 


Percentages  of  Cadets  With  Low  and  Higher  MCET  Scores  in  NELSON-DENNY 
Score  Hanges _ _ _ _ _ — — 


NELSON-DENNY  Score  Range 

MCET  Score  Group _ 25  or  Below _ 26  -  73 _ 76  or  Above 

26  or  Above  12%  (1094)  5)5%  (4966)  55)%  (2931  ) 

25  or  Below _ 66  (1924) _ 32  (  931) _ 2  (  52) 

Note;  The  numbers  of  cadets  are  indicated  in  parentheses. 


Finding  1b;  The  correlations  between  SAT  and  the  achievement  tests 
were  fairly  strong,  but  ACT  and  GPA  were  only  moderately  correlated  with 
the  achievement  tests.  SAT  correlated  with  MCET  and  NELSON-DENNY  .64 
and  .71 ,  respectively  (Table  2).  The  correlations  would  likely  be 
higher  if  only  the  verbal  score  of  SAT  were  analyzed.  In  the  present 
analysis,  only  the  total  SAT  scores  were  available. 

The  correlations  in  Table  2  (r  =  .40,  .45,)  do  not  indicate  strong 
associations  between  ACT  and  the  two  literacy  tests,  possibly  reflecting 
the  fact  that  the  ACT  total  score  consists  of  subscores  for  verbal, 
math,  and  social  and  physical  sciences.  It  should  also  be  noted  that 
the  analyses  involving  ACT  were  primarily  based  on  cadets  attending 
schools  in  parts  of  the  Central,  South,  and  Southwest  geographical  areas 
and  not  a  representative  sample  of  the  entire  ROTC  population. 

The  moderately  low  correlations  between  literacy  skills  and  GPA  in 
Table  2  (in  the  low  .30s  range)  further  suggest  that  academic 
acheivement  is  only  partly  indicated  by  literacy  skills.  Academic  GPA 
has  been  used  as  one  of  the  few  standards  for  contracting  into  ROTC 
Advanced  Course  (U.S.  Amiy,  1986).  Based  on  these  correlations,  GPA 
would  not  be  a  very  direct  method  of  assessing  literacy  skills. 

However,  the  low  correlations  may  be  due,  in  part,  to  variations  in 
grading  standards  across  the  institutions  hosting  ROTC  programs. 

Question  2 


Do  the  literacy  skills  measured  by  MCET  and  NELSON-DENNY  overlap 
with  other  precommissioning  indicators  of  officer  performance?  If  some 
cadets  have  problems  in  literacy  skills,  do  they  tend  to  have  problems 
in  other  aspects  of  officer  training?  Or,  do  strengths  in  other  skills 
compensate  for  their  literacy  problems? 

Finding  2a;  The  relationships  between  the  achievement  test  scores 
and  Military  Science  summary  grades  are  moderately  low.  (See  Table  4.) 
The  correlaltions  between  MCET  and  NELSON-DENNY  and  ROTC  Military 
Science  grade  (.30  ®nd  .29  ,  respectively)  suggest  that  literacy  skills 
have  some  impact  on  the  overall  ROTC  performance  which,  to  a  large 
extent,  may  represent  achievements  in  other  skill  areas. 
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Table  4 


Correlations  Among  Achievement  Test  Scores  and  Other  Precommissioning 


Measures  _ _ _ _ _ — 

MCET 

nelson"- 

DENNY 

GPA 

ROTC 

LNAV 

RIFLE 

PT 

TAX 

ROTC 

.31 

.30 

.47 

LNAV 

.26 

.24 

.18 

.21 

RIFLE 

.05 

.08 

.00 

.03 

.14 

PT 

.03 

-.01 

.12 

.11 

.09 

.05 

TAX 

.09 

.12 

.11 

.10 

.17 

.04 

.09 

PEER 

.10 

.10 

.11 

.21 

.24 

.18 

.24 

.13 

Note:  The 

cases  used  for  analyses 

ranged 

from  7522  to 

25473. 

This  relationship  is  illustrated  further  in  Table  5*  The  majority 
of  cadets  with  MCET  scores  of  25  or  below  were  not  in  the  lowest  25^  in 
terras  of  Military  Science  grades;  ^2%  of  them  were  in  the  highest  25^  of 
Military  Science  grades.  (Note;  The  Military  Science  score  ranges 
refer  to  the  entire  analysis  saraple  and  not  by  individual  institutions.) 


Table  5 


Military  Science  Grades 
for  Cadets  with  Higher 

for  Cadets 
MCET  Scores 

Scoring  25  or  Below  on  MCET  and 
(Percentages  of  Each  MCET  Group) 

Military  Science  Grade 

Range 

MCET  Score  Group 

Lowest  25^ 

Middle  50? 

Top  25? 

26  or  Above 

19?  (1256) 

55?  (3678) 

26?  (1730) 

25  or  Below 

38  (  756) 

50  (  972) 

12  (  235) 

Note:  The  numbers  of  cadets  are  indicated  in  parentheses. 


Finding  2b:  Except  for  land  navigation  (LNAV)  which  showed  some 
overiaF~^th  the  literacy  testa  (.26  and  .24)»  all  other  Advanced  Camp 
measures  yielded  negligible  associations  with  MCET  and  NELSON-PENNY, 
ranging  between  —.01  and  .12.  (See  Table  4 . )  This  is  demonstrated 
further  by  scores  on  the  tactical  application  exercises  (TAX).  As  Table 
6  shows,  most  of  the  cadets  with  MCET  scores  of  25  or  below  were  rated 
as  performing  in  the  higher  ranges  in  terms  of  the  nine  dimensions 
included  in  TAX.  About  one-fourth  of  the  "Low  MCET"  group  ranked  in  the 
highest  25%  for  TAX.  On  the  other  hand,  about  one- fifth  of  the  "Higher 
MCET"  group  ranked  in  the  lowest  25^  range  for  TAX. 
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Table  6 


Tactical  Application  Exercise  Grades  for  Cadets  Scoring  25  or  Below  on 
MCET  and  Cadets  with  Higher  MCET  Scores  (Percentages  of  Each  MCET  group). 

MCET  Score  Group 

Lowest  25^ 

TAX  Grade  Range 
Middle  50^ 

Top  25^ 

26  or  Above 

21^  (1908) 

46^  (4217) 

33?  (3017) 

25  or  Below 

26  (  805) 

50  (1499) 

24  (  718) 

Note;  The  numbers  of  cadets  are  indicated  in  parentheses. 


The  generally  low  correlations  between  the  literacy  tests  and  ROTC 
performance  measures  suggest  that  (a)  the  tests  and  the  performance 
measures  assess  different  skill  areas,  (b)  all  of  these  measures 
collectively  may  yield  a  multi-faceted  assessment  of  cadet  s  officer 
potential  but  (b)  assessment  of  literacy  skills  would  require 
instruments/procedures  specifically  designed  for  that  purpose. 

Finding  2c;  Correlations  among  the  ROTC  measures  (Advanced  Camp 
and  Military  Science  grades)  tended  to  be  low,  ranging  from  .03  to  .24. 
(See  Table  4.)  Correlations  of  these  magnitudes  suggest  that  cadets 
scoring  high  in  one  skill  area  may,  or  may  not,  perform  similarly  well 
in  other  skill  areas,  (it  should  be  noted  that  low  correlations  could 
be  due  to  inconsistent  grading/rating  standards  across  institutions  and 
Advanced  Camps  or  due  to  most  cadets  scoring  in  a  similar,  narrow  range 
of  scores.) 

Question  3 

How  well  do  literacy  test  scores  predict  performance  in  Officer 
Basic  Courses?  If  cadets  score  in  the  low  range  on  tests  of  literacy 
skills,  how  do  they  perform  in  OBC? 

Finding  3;  The  MCET  was  moderately  associated  with  OBC  final 
gradeFTT  the  Armor  and  Infantry  OBCs  (  .47  and  .40,  respectively)  .  (See 
the  first  column  of  Table  7.)  While  these  correlations  do  not  indicate 
a  large  overlap,  they  are  fairly  strong  for  predicting  a  multi-faceted 
performance  measure  from  a  specific  basic  skills  test. 
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Table  7 


Correlations  Between  Precommissioning  Measures  and  Officer  Basic  Course 
Performance _ _ _ _ _ _ 


OBC  Grades 

MCET 

Precommissioning  Measures 
GPA  ROTC  LNAV  RIFLE 

TAX 

PEER 

Armor  OBC 

FINAL 

.47*** 

.24 

.32*** 

.42*** 

.14** 

.14** 

.16** 

Engineer  OBC 
FINAL 

.25* 

.36*** 

.29** 

.45*** 

.12 

.16 

.18 

FIELD 

.06 

.25* 

.06 

.26** 

.15 

.07 

.00 

TRMNG 

-.01 

.17 

-.13 

.29** 

.19 

.01 

.01 

LDSHP 

.14 

.05 

.01 

.26** 

.00 

.26** 

.26** 

Infantry  OBC 
FINAL 

.40*** 

.24*** 

.23*** 

.22*** 

.05 

.23*** 

.28*** 

TRMNG 

.29*** 

.21*** 

.15** 

.19*** 

.01 

.07 

.25*** 

HAINT 

.15** 

.08 

.11* 

.04 

.06 

.15 

.09 

TACTIC 

.25*** 

.13* 

.08 

.14** 

.01 

.19*** 

.18*** 

Note:  Analysis 
ranged  86-116. 

*  2<  *05.  **  . 

samples 
Samples 
£  <  .01  . 

for  AOBC 
for  lOBC 
***  2  < 

ranged  294-389 
ranged  300-350 
.001 . 

.  Samples  for 

• 

EOBC 

These  correlations  may  be  further  examined  in  the  distribution  of 
OBC  final  grades  for  those  who  scored  relatively  low  on  the  MCET  and  for 
those  who  scored  higher.  Table  8  shows  that  about  one-half  of  the  "Low 
MCET"  group  performed  in  the  lowest  25^  range  in  AOBC  and  lOBC  (56^  and 
45^,  respectively).  Almost  half  of  this  group  performed  better  than  the 
lowest  25^  range  in  OBC,  while  some  of  the  "Higher  MCET"  group  were  in 
the  lowest  25^  range  for  OBC  performance. 
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Table  8 


OBC  Pinal  Grades  for  Cadets  Scoring  25  or  Below  on  MCET  and  Cadets  with 
Higher  MCET  Scores  (Percentages  of  Each  MCET  Group  by  School) _ 

OBC  Final  Grade  Range 

MCET  Score  Group _ Lowest  25^ _ Middle  50^ _ Top  25$ _ 


Armor  OBC 


26  or  Above 

22% 

(  49) 

A9% 

(110) 

25  or  Below 

56 

(  38) 

41 

(  28) 

Engineer  OBC 

26  or  Above 

17 

(  12) 

60 

(  43) 

25  or  Below 

21 

(  4) 

50 

(  7) 

Infantry  OBC 

26  or  Above 

13 

(  30) 

56 

(126) 

25  or  Below 

45 

(  34) 

47 

(  35) 

Note:  The  numbers  of  OBC  students  are  indicated  in 


29%  (  67) 

3  (  2) 


23  (  17) 

29  (  3) 


31  (  69) 

8  (  6) 

the  parentheses. 


As  shown  on  Table  7,  the  correlation  between  MCET  and  the  OBC  final 
grades  was  lower  for  the  Engineer  OBC  (EOBC)  (•25)>  which,  based  on  a 
small  analysis  sample  of  86,  does  not  indicate  a  strong  relationship. 

The  correlations  between  NELSON-BENNY  and  the  OBC  grades  were  nearly 
identical  to  the  MCET  correlations  for  all  schools  and  are  not  included 
in  Table  7- 

While  showing  generally  low  relationships  with  specific  course 
grades  in  the  EOBC,  the  MCET  scores  were  somewhat  related  to  the  lOBC 
grades  in  training/management,  vehicle  maintenance,  and  tactics.  Even 
these  correlations,  however,  are  low,  ranging  .15  to  .29.  Without 
additional  data,  the  relationship  between  the  writing  skills  (as 
measured  by  MCET)  and  performance  in  separate  components  of  OBC  training 
remains  inconclusive. 

Table  7  also  shows  correlations  between  the  ROTC  training  grades 
and  OBC  performance,  some  of  which  may  support  the  validity  of  the 
precommissioning  training  and  assessment  for  OBC  req^uirements.  For 
example.  Military  Science  grades  and  land  navigation  produced 
significant  correlations  with  OBC  final  grades  at  all  three  OBCs 
(ranging  .22  to  .45^  For  EOBC,  land  navigation  grades  were 
significantly  correlated  with  all  OBC  subscores  examined,  and  TAX  and 
PEER  correlated  with  OBC  leadership  training  grade.  (The  small  sample 
size  for  EOBC  introduces  some  uncertainty  as  to  the  accuracy  of  these 
correlations.)  For  lOBC,  Military  Science  grade,  land  navigation,  TAX, 
and  PEER  produced  some  significant  correlations  with  OBC 
training/management  and  tactics  grades.  But,  on  the  whole,  the 
correlations  are  low  and  the  results  should  be  considered  as  tentative. 
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Rifle  marksmanship  and  FT  in  Advanced  Camp  did  not  show  much 
relationship  with  individual  differences  in  OBC  performance 
(correlations  for  FT  not  shown  on  Table  7)» 

Question  4 


Given  that  the  OBC  training  involves  diverse  skill  domains,  all  of 
which  are  assumed  to  be  components  of  officer  jobs,  how  well  would  these 
precommissioning  measures  predict  overall  OBC  performance?  What  is  the 
contribution  of  literacy  test  scores  to  this  prediction? 

To  answer  these  questions,  stepwise  multiple  regression  analyses 
(Dixon  &  Jennrich,  1981)  were  conducted.  The  predictors  examined  were: 
MCET,  academic  GPA,  Military  Science  grade  (ROTC),  land  navigation 
(LNAV),  tactical  application  exercise  (TAX),  and  peer  ratings  at 
Advanced  Camp  (PEER).  Rifle  marksmanship  and  FT  were  not  included  in 
the  analyses,  since  they  produced  very  low  correlations  with  OBC  grades. 
The  EOBC  results  are  not  reported  since  they  are  judged  to  be  unreliable 
due  to  an  insufficient  number  of  available  cases  for  this  analysis. 

Finding  4i  The  MCET  score  was  the  best  predictor,  among  the 
precommissioning  measures  examined,  of  the  OBC  final  grades  at  the  Aimor 
and  Infantry  Schools.  Table  9  lists  the  precommissioning  measures  which 
significantly  contributed  to  prediction  of  OBC  performance  (in  the  order 
of  the  amount  of  contribution).  The  predictors  not  listed  did  not  make 
statistically  significant  contributions  in  the  prediction  analyses. 

The  figures  in  the  "Multiple  Correlation"  column  indicate  the 
increases  in  predictive  accuracy  as  each  contributing  measure  is  added 
to  the  analysis.  Since  OBC  grades  are  composite  summaries  of  diverse 
abilities  and  skills,  a  set  of  predictors  would  be  expected  to  include 
measures  of  varied  attributes.  This  variation  was  obtained  among  the 
measures  that  significantly  predicted  AOBC  and  lOBC  performance.  Only 
one  measure  the  MCET  was  common  for  these  two  schools.  Relative 
to  the  other  measures,  MCET  scores  also  made  the  largest  contribution  to 
prediction  of  performance  in  both  schools. 
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Table  9 


Multiple  Correlations  Between  Precommissioning  Predictor  Measures  and 


Precommissioning  Measures 
Predicting  OBC  Final  Grades 

Multiple 

Correlation 

F 

Armor  OBC  (N  =  1 91 ) 

MCET 

.49 

60.05 

LNAV 

.61 

37.11 

ROTC 

.64 

13.96 

Infantry  OBC  (N  =  196) 

MCET 

.34 

25.8b 

GPA 

.38 

6*60 

PEER 

.43 

8.62 

Note;  All  measures  analyzed  were 

standardized . 

Discussion 


The  major  findings  will  be  summarized  with  reference  to  the  issues 
pertinent  to  the  Task  Force  mission. 

1.  For  the  entire  analysis  sample,  cadets'  performance  on  two 
standardized  tests  of  literacy  skills  were  similar.  Thus,  the  pattern 
of  findings  based  on  Missouri  College  English  Test  or  Nelson-Denny 
Reading  Test  may  generally  apply  to  other  standardized  tests  of  literacy 
skills. 

However,  on  an  individual  level,  cadets'  score  on  one  test  may  not 
necessarily  coincide  with  the  same  level  of  performance  on  the  other 
test.  Establishment  of  standards  for  literacy  skills  diagnosis  should 
be  based  on  distribution  of  scores  on  the  particular  test  to  be  used, 
collected  from  a  representative  sample  of  cadets. 

2.  The  achievement  tests  and  ROTC  training  measures  were  not 
highly  correlated.  Collectively,  all  these  measures  may  provide  a 
method  to  assess  various  aspects  of  cadets'  officer  potential.  However, 
assessment  of  literacy  skills  would  require  methods  specifically 
designed  for  that  purpose.  The  correlations  also  suggested  that,  with 
the  "whole  person"  assessment,  some  cadets  with  deficiencies  in  some 
skills  may  achieve  higher  ratings  for  other  skill  areas. 

5.  The  writing  and  reading  abilities  as  measured  by  MCET  and 
NELSON-DENNY  were  found  to  be  related  to  overall  performance  at  AOBC  and 
lOBC.  The  correlations  on  Table  7  show  that  cadets  scoring  relatively 
low  on  the  tests  also  tend  to  perform  in  the  lower  range  in  OBCs  more 
than  cadets  scoring  higher  on  the  tests. 

The  correlations  between  MCET  and  OBC  final  grades  may  be 
considered  fairly  sizable  in  the  traditional  job  selection  framework. 
That  is,  even  with  correlations  in  the  .40s  range,  knowing  cadets' 
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scores  on  these  tests  would  improve  the  accuracy  of  selecting  those  who 
are  likely  to  do  well,  say,  in  OBC  training. 

However,  increasing  the  selection  efficiency  is  currrently  not  the 
primary  objective  of  the  Task  Force.  The  precommissioning  literacy 
standards,  when  established,  are  intended  to  identify  cadets  who  might 
need  developmental  teaching  in  order  to  perform  junior  officer  tasks 
effectively.  In  this  context,  the  correlations  in  the  .40s  range  also 
mean  that  about  one-half  of  cadets  in  the  lowest  quartile  of  MCET  scores 
performed  at  higher  levels  with  regard  to  overall  OBC  training.  For 
many  of  these  students,  strengths  in  skills  not  measured  by  these  tests 
or  their  effort  may  have  compensated  for  their  writing/ reading  problems. 
On  the  other  hand ,  it  is  plausible  that  their  OBC  and  subsequent  job 
performance  might  be  enhanced  even  more  with  improvements  in  their 
literacy  skills. 

Aside  from  the  need  to  refine  the  assessment  standards,  the  outcome 
of  the  pilot  project  needs  to  be  examined  with  regard  to  the 
social/political  ramifications  of  the  program.  As  described  in  an 
earlier  report  (Hunter,  1986),  gender  differences  in  the  MCET  and 
NELSON-DENNY  scores  were  minor,  but  there  were  performance  differences 
by  racial/ ethnic  groups.  The  use  of  any  standardized  literacy  skills 
test  will  probably  lead  to  an  overrepresentation  of  minority  ethnic 
groups  among  the  cadets  recommended,  or  required,  to  receive 
developmental  teaching.  Whether  or  not  this  would  constitute  an  adverse 
(i.e.,  negative)  impact  would  depend  on  the  design  and  implementation  of 
the  literacy  standards  program.  Will  the  developmental  teaching  create 
a  serious  overload  in  cadets'  time  commitments?  Will  being  identified 
as  needing  additional  training  leave  a  carry-over  effect  in  the  overall 
cadet  evaluation,  even  if  the  literacy  skills  are  improved?  If  this 
type  of  problems  are  avoided,  then  developmental  training  should  not 
hurt  anyone  whose  literacy  skills  fall  markedly  below  the  average  range 
of  his/her  peers.  Whether  or  not  the  proposed  program  may  create  an 
adverse  {i.e.,  not  differential  per  se,  but  negative)  impact  on  some 
subgroups  also  is  an  issue  to  be  evaluated  in  the  course  of  the  pilot 
project. 


Conclusion 


Standardized  literacy  tests  may  be  one  of  the  better  predictors  of 
OBC  performance,  possibly  due  to  the  uniformity  in  testing  and  scoring. 
There  seems  to  be  some  tendencies  for  cadets  scoring  in  the  low  range  on 
a  standardized  literacy  skills  test  to  perform  relatively  poorly  in  OBC. 
Standardized  tests  would  provide  consistency  in  the  literacy  skills 
screening  administered  across  highly  heterogeneous  institutions. 

On  the  other  hand,  the  accuracy  of  the  assessment  technology  is 
such  that  neither  standardized  tests  nor  other  precommissioning  measures 
precisely  predict  OBC  performance.  Currently,  there  is  no  systematic 
basis  by  which  to  determine  which  test  score  would  consistently 
differentiate  poor  and  high  performance  in  OBC.  Furthermore,  all 
subjects  examined  for  the  OBC  analyses  did  graduate  from  OBC,  and  we  do 
not  have  information  on  how  relatively  low  performance  in  OBC  is  related 
with  subsequent  job  performance. 
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In  order  to  establish  sound  precominissioning  literacy  skills 
standards,  much  more  information  is  needed.  The  pilot  project  should 
provide  a  valuable  means  by  which  to  refine  the  assessment  procedures, 
evaluate  the  effectiveness  of  developmental  teaching,  examine  the 
relationships  between  the  assessment  measures  used  and  junior  officer 
performance,  and  investigate  the  possibility  of  differential  impact  of 
the  program  on  subgroups. 
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